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A key question in pliysics education is tlie effectiveness of tiie teaciiing metiiods. A curriculum 
tiiat has been investigated at the University of Central Florida (UCF) over the last two years is the 
use of particular elements of The Physics Suite. Select sections of the introductory physics classes 
at UCF have made use of Interactive Lecture Demonstrations as part of the lecture component of 
the class. The lab component of the class has implemented the RealTime Physics curriculum, again 
in select sections. The remaining sections have continued with the teaching methods traditionally 
used. Using pre- and post-semester concept inventory tests, a student survey, student interviews, 
and a standard for successful completion of the course, the preliminary data indicates improved 
student learning. 
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way to a STEM degree. 
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The University of Central Florida (UCF) is a large 
metropolitan university located in Orlando, Florida. 
Shortly before this study was started, the U.S. Depart- 
ment of Education National Center for Education Statis- 
tics ranked UCF as the sixth largest university in the 
country based on a Fall 2006 enrollment of 46,646 stu- 
dents fl|. Since that time, the university's enrollment 
has grown to over 55,000, making UCF the third largest 
university in the United States going into the Fall 2009 
semester 0, Q . The calculus-based introductory physics 
courses. Physics I (Newtonian Mechanics) and Physics 
II (Electricity and Magnetism with some Optics) are 
required for all students of the College of Engineering 
and Computer Science. This college alone had an en- 
rollment of over 6000 in the spring of 2009 The 
large number of students going through the introduc- 
tory classes each semester, along with the students from 
the College of Sciences, which includes the Physics De- 
partment, and a large contingent of students from other 
colleges taking the department's physical science class 
make UCF's Physics Department the second- largest in 
the country based on the number of credit-hours taught 
per semester The teaching of these introductory 

classes is a large-scale operation, with ten to twelve 
lecture sections of eighty to over one-hundred students 
taught each semester, along with twenty to twenty- two 
lab sections of thirty to thirty-five students each. Reten- 
tion of the students within the majors that make up the 
science, technology, engineering and mathematics fields 
(STEM) 2S a challenge for UCF as well as universities na- 
tionally [6|. The introductory physics courses are one of 
the first large hurdles that students must clear on their 
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There are many innovative approaches to teaching in- 
troductory physics (see Ref. [3] and references therein). 
One that has been demonstrated to effectively improve 
students' conceptual learning is curriculum presented in 
The Physics Suite 0, [1] . Some elements of The Physics 
Suite apply readily to the large-scale approach to the 
teaching of the introductory classes at UCF, with little 
or no change to the scheduling format or the infrastruc- 
ture used by the department. Interactive Lecture Demon- 
strations (ILD) ^ can be used effectively in a large lec- 
ture hall setting, so they can be implemented in any 
of the sections scheduled at UCF. Similarly, RealTime 
Physics (RTP) [lll - [l3l | is designed for use in a lab setting 
with groups of two to four students performing experi- 
ments using computers and probeware, much of which 
is already used in the UCF introductory physics labs. 
These elements of The Physics Suite bring physics edu- 
cation research-based curriculum into an existing large- 
scale physics instruction program while maintaining the 
department's existing method of scheduling and class 
structure and making use of the department's existing 
equipment. 



The evaluations outlined in this paper provide evidence 
of improved student learning due to the effectiveness of 
including The Physics Suite methods as a major part 
of the instruction in the introductory physics classes. 
The large scale of UCF's introductory physics program 
has allowed a comparison of conceptual learning between 
significant numbers of students taking part in the new 
method of instruction with those in traditional instruc- 
tion, as well as those with partial involvement in both 
methods, over a short span of time. The number of sec- 
tions of each class available for this survey has also pro- 
vided the opportunity to study the relative benefits of In- 
teractive Lecture Demonstrations and RealTime Physics 
when implemented seperately as well as in concert. 
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II. INTRODUCTORY PHYSICS CLASSES AT 
UCF, AND CHANGES MADE TO IMPLEMENT 
THE PHYSICS SUITE 

The lecture sections of the introductory physics classes 
at UCF are taught in an auditorium-style lecture hall 
seating 80-120 students. Traditional format lectures are 
taught in a visual method determined by the instructor 
— that is, either projected powerpoint slides or flash- 
ing of transparencies or blackboard/ whiteboard presen- 
tation. A few instructors have made occasional use of 
electronic student response systems (ERS) to incorpo- 
rate some active student participation into the format 
and/or have used a combination of the visual methods 
for the delivery of the lectures. 

The sections chosen for participation in The Physics 
Suite project used Interactive Lecture Demonstrations 
during one lecture class each week. A demonstration re- 
lating to the current topic being addressed in the class 
is set up. Students are asked to make a prediction of 
what results will be seen during the demonstration. The 
predictions are recorded as a poll, using the ERS. Stu- 
dents are then encouraged to interact in small groups, 
discussing their predictions and explaining their reason- 
ing. After the discussion interval, students are polled 
again and the results are displayed. The demonstration 
is then conducted and students can see the actual results. 
Finally, the physical principles involved are discussed, 
with participation both from students who had reasoned 
correctly and from some who had based their answers on 
incorrect preconceptions. This approach of direct con- 
frontation of preconceptions with actual demonstrations, 
requiring the students to thoroughly reason through the 
problem both before and after the demonstration has 
been shown to be an effective teaching method. It de- 
velops deeper understanding by connecting students' in- 
tuition to the concepts addressed, and it is effective in 
changing flawed preconceptions [ij, [H, [13, US • 

The lab component of the introductory classes is a sep- 
arate, one credit-hour class. The classrooms are arranged 
for eight stations of four students each, and each station 
equipped with a computer and Pasco CI-6560 interface. 
The experiments used in the traditional format lab are 
mostly "cookbook" -style. The students present their re- 
sults in a report composed outside of class and turned in 
the following meeting. The format of the report is deter- 
mined by the instructor. Part of the focus of the manual 
is on experimental set-up and method, and there is some 
introduction to error analysis. 

The lab sections using RealTime Physics as their lab 
manual are set in the same rooms and make use of much 
of the same equipment as the traditional format labs. 
The RTP manual makes use of a guided-inquiry approach 
to the execution of the experiments. After a brief out- 
line of the experimental set-up, the students are asked 
to make predictions of the results they expect to see. 
Like the ILDs in the lecture sections, this prediction step 
will target preconceptions and lead to discussion of the 



underlying principles among members of the lab group. 
For most activities, the data is plotted in real time, as 
the experiment takes place. The students are encouraged 
to make multiple runs of each experiment, watching the 
relation between the physical activity and the mathe- 
matical plot. The actual results are compared with the 
predictions made earlier. Questions are asked to get the 
students to think about the physical principles determin- 
ing the experimental results. Some mathematical analy- 
sis is done with the graphs and data from the experiment 
to show relations between physical quantities. 

An example of an activity performed during one of 
the one-dimensional motion labs from Physics I follows: 
A student holds a ball above a motion detector that is 
hooked to a computer. The students in the group pre- 
dict what will happen to velocity and acceleration for 
the ascent, turning point, and descent of the ball thrown 
vertically. Then the ball is thrown in the air so the mo- 
tion detector can plot velocity versus time and acceler- 
ation versus time. Now students discuss the results and 
can see that even though velocity changes, acceleration is 
constant. The plots of velocity and acceleration are com- 
pared, illustrating the mathematical relation between the 
quantities. Careful attention to the graphs also helps il- 
lustrate the vector nature of the quantities (2ll - [23j . 

The lab report is completed within the lab class time, 
consisting of answering the questions posed during the 
activities. There is a set of homework questions follow- 
ing each lab, designed to reinforce concepts developed 
through the activities in the lab. Through the Fall 2007 
semester, the first semester of the project, the homework 
was optional. Since then, the students have been re- 
quired to turn this in at the start of the following lab. 
The emphasis of the RTP labs is on understanding the 
physical concepts addressed and interpreting the graphs 
of mathematical relations followed by the physical quan- 
tities [Hill. 



HI. METHOD OF EVALUATION 

To evaluate the effectiveness of these portions of The 
Physics Suite method at UCF, all the students in the 
introductory physics classes were given a pre-test dur- 
ing the first meeting of the lab section, and a post-test 
on the last day of the lab. The tests used are concept 
inventory-style tests and the same tests were given to 
all students both times. The Physics I test is the Force 
and Motion Concept Evaluation and is included as an ap- 
pendix to the Thornton/Sokoloff article cited [10]. The 
Physics II test is the Electric Circuits Concept Evalua- 
tion, also created by Sokoloff and Thornton. The dif- 
ference between each student's score between the pre- 
and post-tests was not the score evaluated. Instead, stu- 
dents' pre- and post-test scores were compared based on 
the percentage of material that they learned during the 
semester. The improvement score, normalized gain as 
introduced by Hake, calculates what percentage of the 
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questions missed on the pre-test are answered correctly 
on the post-test. The formula used to calculate the nor- 

maliypH crain iq Oain — (-Pos* Correct)-(Pre Correct) -, 
manzea gam is t^am - (per/ect Score)-{Pre Correct) ^ 

m 

The students in each semester registered for their lec- 
ture and lab classes without knowing if either would in- 
clude a component of The Physics Suite, so their place- 
ment was random. A small number of students each 
semester take either the lab or lecture component alone. 
The students taking only the lecture component did not 
take the concept evaluation tests. 

The scores from the pre- and post-tests were grouped 
into four categories: students taking an ILD lecture sec- 
tion and an RTP lab, students taking an ILD lecture 
section and a traditional format (TF) lab, students tak- 
ing a traditional format (TF) lecture and an RTP lab, 
students taking a TF lecture and a TF lab. Students 
who missed either test due to add/drop of a class or an 
absence on either test day were not counted. Also, for 
any student who had taken a pre-test and switched sec- 
tions and took it again the lower score was kept. These 
statistics were used to evaluate the effect of The Physics 
Suite elements on conceptual learning. 

The students' successful completion of the course, de- 
fined as attaining a "C" or better, is compared among 
the four categories. The percentage of each category that 
successfully completed the course during the study is pre- 
sented for both Physics I and Physics II. 

Finally, student opinions were solicited. Inter- 
views were conducted with student volunteers after two 
semesters, the Spring 2008 semester and the Spring 2009 
semester. The students did not receive any monetary or 
grade compensation for the interviews. In the spring of 
2008, twenty-eight students were interviewed, eight stu- 
dents from Physics I and twenty from Physics II. Six of 
the students were Physics II RTP students who had taken 
Physics I during the fall with the RTP method. A survey 
was given to all sections of the lab classes at the end of 
the Spring 2009 semester. Excerpts from the interviews 
and results from the survey are presented. 
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FIG. 1: Cumulative Physics I results 
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FIG. 2: Cumulative Physics II results 



IV. EVALUATION: CONCEPT TEST RESULTS 

The students' conceptual learning as measured by the 
performance on the Force and Motion Concept Evalu- 
aizon(FMCE) and Electric Circuits Concept Evaluation 
(ECCE) pre- and post-tests is presented first. The com- 
parison is made between students taking sections using 
either Interactive Lecture Demonstrations (ILD) in their 
lecture, RealTime Physics{WrP) in their lab, or both 
components of The Physics Suite{IL'D & RTP) and those 
taking only traditional format classes (TF) in both their 
lecture and lab component. 

Figures [T] and [2] show the two-year cumulative perfor- 
mance in Physics I (PHY2048) and Physics II (PHY2049) 
separately. The gains for all formats are higher in Physics 
I, with students taking both ILD and RTP components 



gaining over 50% compared to 19% for students having 
both components in the traditional format. In Physics 
II, the gains are less for all formats, but students taking 
any component of The Physics Suite more than doubled 
their gain over the TF-only group. 

The remaining figures for this section show the perfor- 
mance of students in the introductory classes by semester 
for the first two years of the project, from the most re- 
cent results back to the start of project. Note that in 
PIIY2049 there are no students in the TF lab categories 
after the first semester of implementation. After positive 
results in the initial semester of the project, all sections of 
the Physics II lab component were switched to the RTP 
format. 

The latest results from the Spring semester of 2009 are 
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FIG. 3: FMCE & EECE results for Physics I & II from Spring 
2009 
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FIG. 5: FMCE & EECE results for Physics I & II from Spring 
2008 
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FIG. 4; FMCE & EECE results for Physics I & II from Fall 
2008 
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FIG. 6: FMCE & EECE results for Physics I & II from 
Fall 2007. This was the only semester to include TF labs 
in Physics II. 



shown in Figure [31 Figure 0] shows the resuhs from the 
Fall semester of 2008. Figure [S] shows the performance 
in the Spring of 2008. Finally, Figure [6] shows the Fall of 
2007, the first semester of implementation. 

For Physics I, the number of students in each category 
and for each semester is given in Table HI The Physics II 
numbers are in Table HIl Fall 2007 was the only semester 
with traditional format labs in Physics II. 



Physics I Numbers by Semester 

Semester ILD&RTP ILD&TF TF&RTP TF&TF 



Fall 2007 


26 


57 


72 


270 


Spring 2008 


15 


42 


43 


186 


Fall 2008 


19 


49 


60 


104 


Spring 2009 


32 


121 


34 


120 


Total 


92 


269 


209 


680 



TABLE I: The number of students taking the FMCE, by 
semester. 
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Physics II Numbers by Semester 

Semester ILD&RTP ILD&TF TF&RTP TF&TF 



Fall 2007 


17 


21 


66 


191 


Spring 2008 


15 




141 




Fall 2008 


46 




210 




Spring 2009 


78 




210 




Total 


156 


21 


627 


191 



TABLE II: The number of students taking the EECE, by 
semester. 
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FIG. 8: Percentage of student responses to the statement: 
"The experiments performed in the lab have caused me to 
change my mind about how some physical process works." 



a higher withdrawal rate. At this point the investigation 
has not differentiated between withdrawals and failures. 
The retention issue is an important one and the other 
apparent successes of the project make it seem that it 
should help with this also. 



FIG. 7: Percentage of students successfully completing the 
introductory physics classes (Fall 2007-Spring 2009) 



EVALUATION: SUCCESSFUL CLASS 
COMPLETION 



The FMCE and ECCE test conceptual understanding. 
Another important measure of the success of a course is 
given by the percentage of students that successfully com- 
plete the course. In the case of the introductory physics 
courses, the majority of the students are from the College 
of Engineering and Computer Science, which requires a C 
or better in the core courses. Figure [7] shows the percent- 
age of students attaining at least a C grade in Physics 
I from the Fall 2007 semester through the Spring 2009 
semester. Again, the performance is divided into the 
categories based on the combination of lecture and lab 
instruction method. This is the one area where taking 
both ILD and RTF components has not shown to be an 
advantage. The effect may be masked by less grade infla- 
tion in the ILD classes taught by the project's PI(C.E.), 
and by the stricter rules of student evaluation he has es- 
tablished in the RTF labs. Since the combination of the 
two elements of The Physics Suite created the largest 
gain measured in conceptual understanding, it may be 
that it leads to a bigger split between the successful stu- 
dents and those that are still struggling. This may lead to 



VI. EVALUATION: STUDENT INPUT 

The evaluation also involved student input. A survey 
was given to all students taking the post-test at the end of 
the Spring 2009 semester. The survey consisted of fifteen 
statements relating to the benefits of the lab component 
of the course, the correlation between the lab and lec- 
ture components, the recitation sessions conducted dur- 
ing the lab meeting time, and possible ways to maximize 
the benefit of the lab component. Students were asked to 
rank their agreement with each statement on a five-point 
scale: strongly agree, agree, neutral, disagree, or strongly 
disagree. 

Here the responses of Physics I students are shown for 
two statements that are key to both the motivation for, 
and the success of the implementation of The Physics 
Suite. They refer to the goal of challenging common pre- 
conceptions held by students of the introductory physics 
classes, which The Physics Suite curriculum is written 
specifically to address, and the goal of better coordinat- 
ing the lab component topics with the lecture component 
topics chronologically during the semester. 

Figure |S] shows the responses to the statement, "The 
experiments performed in the lab have caused me to 
change my mind about how some physical process 
works." The students who had taken a RealTime Physics 
lab (RTF) had a higher percentage of positive responses 
than those taking the traditional format lab (TF). The 
strongest positive response came from students taking 
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FIG. 9: percentage of student responses to the statement: 
"There is a connection between the concepts addressed by the 
lab experiments and the theory covered in the lecture class." 

the RTP lab, with the traditional lecture component. 
These students seemed to have a strong sense that the 
lab exercises contributed to their understanding of the 
physical principles involved. The responses of the stu- 
dents taking an RTP lab component with an ILD lec- 
ture, although still positive, were not as strong as the 
RTP lab with traditional lecture students. This is the 
group that performed the highest on both the concep- 
tual test and in the lecture component grade. The lower 
positive response may be due to the wording of the state- 
ment. These students may feel that their preconceptions 
were challenged during the ILD lecture as well as during 
the RTP lab experiments. 

Figure |9] shows the percentage of the responses to the 
statement, "There is a connection between the concepts 
addressed by the lab experiments and the theory covered 
in the lecture class." Again, students have responded 
more positively if they had the RTP lab than if they had 
a TF lab. 

Student volunteers were interviewed following both the 
Spring 2008 and Spring 2009 semesters. The earlier in- 
terviews have been transcribed and the student feedback 
is summarized here. 

On major items, such as 'Do you like the format?', 
'Have the experiments changed misconceptions?', and 
'Do the experiments help you learn physics?', the ma- 
jority of students gave positive feedback. The interac- 
tive nature of the RTP labs was praised by a number 
of students. Many students claimed to have had pre- 
conceptions modified by conducting the experiments in 
the RTP lab. A question in the interview addressed the 
homework and most students had a positive response. It 
was an indication the homework was fulfilling its intended 
purpose. The interviews were conducted in a focus group 
setting. Examples of two questions and typical student 
responses follow. The responses have been paraphrased 



to accommodate grouping of like responses. 

1. Do you like this (overall) format as com- 
pared to a traditional lab format where you 
take all the data and later write a report? 
Why? 

- Much more interactive 

- More effective because you forget when you 
try and write the report at home 

- Somewhat, explains to a point that makes you 
find out a purpose 

- Good to answer questions as going through 

- Does more to reinforce 

2. Do the lab report questions help understand 
the material? 

- Helps understand 

- Makes you stop and think 

- Helps if you take your time to do it 

- Observing then writing reinforces 

- Seeing the result really helps with concept 

VII. IMPLEMENTATION ISSUES 

Although the students are encouraged to do their best 
on the pre- and post-tests, and most do, there are some 
who just quickly mark down an answer. There needs to 
be an incentive for all to try their best. It cannot be 
directly related to their grade because, in the case of the 
pre-test, it would be unfair to grade students on mate- 
rial they have not seen. The graduate teaching assistants 
have been asked to offer some form of extra credit to stu- 
dents as an incentive to give a full effort on both tests. 
Some methods that have been tried are rewarding the 
top-scoring students on the pre-test, rewarding both the 
top scoring and highest normalized gain students on the 
post-test, and counting the post-test as a full-score re- 
placement quiz grade based on effort (as judged by the 
instructor). 

Another issue is that as time goes on students from 
the past RTP labs will have corrected homework that 
they can share with current RTP students. Although 
this would not show directly in the pre/post test results, 
some students may not put full effort into learning. A 
possibility may be to have different homework sets ap- 
pended to those in the lab manual. 

Attaining a good correlation between the lab and the 
current topic being covered in the lecture component is 
always a challenge and was part of the motivation for 
undertaking the project. Much of the problem involves 
the different pace of the various lecture instructors. The 
RTP labs for Physics I have been in fairly good agree- 
ment with the lectures. Physics II has been more diffi- 
cult. The labs start immediately with circuits, while the 
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lecture deals with static charges, electric force and elec- 
tric fields. There is also a point where the lecture starts 
addressing magnetism and there are no corresponding 
labs. The geometric optics labs have been inserted into 
the lab curriculum at this point, and then circuits are re- 
sumed once the lecture begins with some introduction of 
induction. Although students have had some complaints 
about the disconnect between the two components, most 
have been glad for the experience with circuits when that 
topic is reached in lecture. 

VIII. CONCLUSION 

There is strong evidence, based on the results from 
the pre- to post-test gains on the FMCE and ECCE, 
that The Physics Suite elements implemented into the 
introductory physics classes at UCF have a positive ef- 
fect on students' conceptual learning. This is especially 
true when both the Interactive Lecture Demonstrations 
and RealTime Physics labs are used in conjunction. How- 
ever, the investigation shows evidence that either com- 
ponent benefits student learning when used alone. This 
makes The Physics Suite a good option for a large-scale 
physics program with many different instructors. Real- 
Time Physics as a stand-alone method for lab instruc- 
tion pairs well with the lecture component of the course, 
whether or not the lecture instructor chooses to include 
Interactive Lecture Demonstrations as part of the course. 

This improved performance does not seem to have af- 
fected the number of students who successfully complete 
the courses, although those that do succeed are doing 
so at a higher level. The students that withdraw or fail 
often do not attend class, or do not put any efi^ort into 
their studies of the subject. Even if the presentation of 
the material is improved, it does not have an effect on 
this group. The effort to stimulate learning in the lowest 



performing students is an area that will require further 
focus. 

There is also evidence that the students themselves feel 
that the approach of The Physics Suite has positively 
impacted their learning. Responses to the survey ques- 
tions and statements made during the interviews indicate 
a perception among the students that the inquiry-based 
activities of RealTime Physics and the active participa- 
tion in lecture required in Interactive Lecture Demonstra- 
tions play a large role in helping them grasp the difficult 
material of the introductory physics classes. 

The success of the project through the first two years, 
based on the evaluations done and also on positive feed- 
back from the students themselves indicates that there is 
good reason to continue with, and expand, the implemen- 
tation of The Physics Suite at The University of Central 
Florida. 
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